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ABSTRACT 

This project v?as designed to secure and screen 
available education materials appropriate for actua-lization of 
comprehensive Career Education Model-1 (CCEM-1) objectives, and to 
process, assess, and classify selected potential commercial education 
materials for incorporation into instructional units and/or 
supportive educational activities. To achieve the objectives, a 
50-member consulting team was assembled from both the educational and 
business communities, and this team: (1) identified suppliers, (2) 
secured suppliers participation, (3) identified career educational 
materials, (4) procured suitable materials^ (5) assessed suitable 
materials, (6) indexed and abstracted the positively assessed 
material, and (7) delivered the materials and instruments. Of the 580 
commercial suppliers contacted, 313 had materials available for 
procurement and assessment, and 128 of these reported additional . 
materials under development. A total of 2,016 of the 2,344 units 
received were assessed as applicable to ccEM-1 objectives. Of the 
assessed materials, 22 percent was applicable to Grades K-6, 27 
percent to Grades 7-9, 36 percent to Grades 10-12, and 13 percent to 
Grades 13-14. These and other findings are discussed, and sample 
assessment instruments are included. (SB) 
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CEBXrFTED PUBLIC ACCOIIXTA^'TS 
232 SOUTH HIVEHSIDE PLAZiL 
CHICAGO. ILLINOIS 60S06 



Dr. Aaron J. Miller, Director 
Comprehensive Career Education Model I 
The Center for Vocational and Technical 

Education 
The Ohio State University 
2501 North Starr Road 
Columbus, Ohio 43221 



Dear Dr, Miller: 

With the submission of the attached report, Peat, Marwick, Mitchell & Co. 
(PMM&Co.) has completed its engagement to identify, procure, classify, assess, index and 
abstract commercially produced career education materials that meet the requirements of 
the Comprehensive Career Education Model I (CCEM-I). As required by contract, the 
following acknowledgment is cited: 



The project presented or reported herein was performed pursuant with a grant 
from the U. S. Office of Education, Department of Health, Education and Wel- 
fare. However, the opinions expressed herein do not necessarily reflect the posi- 
tion or policy of the U. S. Office of Education, and no official endorsement by 
the U. S. Office of Education shall be inferred. 



This letter presents a project summary, the findings which resulted from the 
project, our conclusions and recommendations, and an overview of the attached final 
report. 



PROJECT SUMMARY 

The engagement was completed in compliance with PMM&Co.'s response 
(dated June 19, 1972) to your Request for Proposal (RFP) No, CEP.2A. Work was 
started on July 5, and the draft of the final report, including appendixes, was submitt^<^ 
on November 10, 1972. 
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The project was an integral part of the efforts of the U. S. Office of Education, 
The Center for Vocational and Technical Education (CVTE) at The Ohio State Univer- 
sity, and six Local Education Agencies (LEA) selected by the USOE. CVTE was assigned 
as prime grantee to develop, test and install the school-based Comprehensive Career Educa- 
tion Model (CCEM); the LEA's were to work coopenitively in developing appropriate 
program components for a validated, nationally transportable career education pro-am 
available to all public schools. 

Specific objectives of the engagement were to: 

Secure and screen all available commercial education materials 
appropriate for a comprehensive career education program as it 
relates to specific instructional units 

Process, assess and classify selected potential commercial educa- 
tional materials for incorporation into instructional units and/or 
supportive educational activities 

Document activities and submit required reports and informa- 
tion. 

A total of 50 people were included in the project team. Under the direction of 
a project director, work was divided into five major functional activities: field survey, 
assessment, quality control, systems development, and support and administration. In 
addition, outside legal counsel was engaged, and the overall project was monitored by a 
five-man team comprised of senior PMM&Co. partners. 26 members of the project team 
were professional personnel residing in the Columbus, Ohio area. These individuals were 
recruited and trained in assessment, indexing and/or abstracting functions. Hence, 
CCEM-I has available in its geographical area a pool of talent for continuing this effort if 
desired. In total, more than 10,800 manhours were devoted to the project in the four- 
month period. 



PROJECT FINDINGS 

The following major findings resulted from this project: 

Of the 580 commercial suppliers contacted, 313 had materials 
available for procurement and assessment 

Of the 580 suppliers, 128 that had materials available reported- 
ly have additional career education materials under development; 
35 of the 267 that reported no materials available at this time 
also have materials under development 

More than 33,600 pieces of material grouped into 3607 units 
were identified and approximately 2700 units were requested 
for assessment 

Of the 2344 units received, 2016 (more than 12,800 pieces of 
material) were assessed as applicable to CCEM-I objectives 
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Of assessed material 22 percent was applicable lo grades K-6, 
27 percent to 7-9, 36 percent to 10-12, and 13 percent to 
13-14 

Of the material assessed, 54 percent was printed and 32 perv'ent 
was film; the remainder was classified a.s manipulative and 
audio-visual 

66 percent of the assessed material cost less than $100 

Career education units are available ccmmercially for all grade 
levels - a low of 250 for use in kindergartens to nearly 1750 for 
grade 12 

More than 1000 units were judged appropriate for four instruc- 
tional strategies: 

- discussion - 1709 

- self-instruction - 1357 

- group or panel discussions - 1237 

- individual or group counseling - 1018 

Of the 2016 positively assessed units, 649 were determined 
applicable for individualized, small group or large group instruc- 
tion. Only two units were deemed applicable to community re- 
lations and community agency support programs 

The appropriateness (high, average, low) of units to the CCEM 
matrix by grade level and element (self awareness, educational 
awareness, career awareness, economic awareness, decision^ 
making, beginning competency, employability skills and 
attitudes and appreciations) is as follows: 

• Most self awareness units are for grades 
9-12; 50 percent were judged average 

- Educational awareness units are most ap- 
plicable to grades 742; 15-20 percent were 
judged high 

- At least 195 career awareness units are available 
for each grade level; more than 50 percent of 
those for grades 4, 5 and 6 were judged low 

- At least eight economic awareness units are 
available for each grade from K-12; nearly 50 
percent of the units for grades 5 through 9 were 
judgfed low 

- Only 136 un:ts for decision-making were judged 
applicable, most for grades 11 and 12. Of the 21 
deemed appropriate for grades 1 through 6, ap- 
proximately 50 percent were judged high 
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- Most determinations of applicability for beginning 
competency were for grades 10-14: none of those 
for K-9 were judged high 

- Of the nearly 570 units determined appropriate 
for employability skills. 262 were judged applic- 
able for grades 1 1 and 12: only 86 in total and 
50 for grades 11 and 12 were judged high 

- At least 100 units were deemed applicable to the 
attitudes and appreciations element for each 
grade level: the percent judged high is signify 
icantly lower than for other elements where 
large numbers of units are applicable 

The applicability (high, average, low) of units to thirteen target 
areas (curriculum, career information, career preparation, guid- 
ance, staff development, placement/guidance, evaluation, sup- 
port systems, community relations, post-secondary 13-14, rural 
education, management, and other), is as follows: 

- Cf the nearly 6000 judgments of applicability, 
more than 3566 were in the career information 
target area; of these, more than 850 were judged 
high 

- 500 or more units were also judged applicable 
for curriculum (569), career preparation (614), 
and guidance (1129); only 41 positive judgments 
were made for the other nine target areas 

- The element most frequently indicated was 
career awareness, followed by attitudes and 
appreciations 

Overall judgments (high, average, low) of the suitability of 
positively assessed units to CCEM-I needs for materials are as 
follows: 

- 40 percent of the determinations of occupational 
focus were judged high 

- 74 percent were judged low for innovativeness 
(substantive) 

- Nearly G3 percent were rated low for innovative- 
ness (procedural) 

- Approximately 68 percent were deemed low for 
participant involvement. 
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CONCLUSIONS AND RECOMMENDATION S 

Project findings indicate that commercial suppliers are making great efforts to 
provide career education materials. However, as our findings show, the number of units is 
relatively limited for several elements and target areas for specific grade levels. In addition, 
judgments (high, average, low) of the applicability and suitability of units lo CCEM-I re- 
quirements show that quality can be improved. However, considering the relative newness 
of career education as defined by USOE and detailed by CCEMJ, commercial suppliers 
should be commended for their efforts to be responsive to the market. 

Results of this engagement and its companion project (to identify and assess 
non-commercial materials; completed in January, 1972) prov/de CCEM officials with the 
most complete existing inventory of available career education materials. This inventory 
should be extremely valuable to users in identifying available products and their sources. 
This information should also be of great assistance to materials suppliers who are interest- 
ed in improving their products and filling gaps in units required to meet CCEMJ objec- 
tives. To facilitate these potential applications of the existing inventory and to maximize 
the results of the substantial expenditure for the two projects, USOE and CCEM officials 
should place top priority on: 

Disseminating iniormation regarding the availability of career 
education materials to consumers through the ERIC system 

Continuing the evaluation process to keep the inventory up-to- 
date. 

Specific recommendations are to: 

1. Place the results of the project(s) in the ERIC system. 

2. Establish and make available necessary resources for continuing the process 
of identifying, procuring, assessing, abstracting and indexing career education materials. 

3. Place high emphasis on screening materials through careful review of 
catalogs to determine their probable appropriateness before issuing Material Procurement 
Requests. This will substantially reduce the resource and time requirements associated 
with each positively assessed unit. 

4. Adopt and implement systems used in this project and make modifications 
and refinements deemed necessary to keep them efficient and operable at the least re- 
quired cost. 

5. Maintain and expand relationships with suppliers as appropriate. This re^ 
quires prompt 3nd responsive haadling. 

6. Do not change the CCEM matrixes or definitions for at least one year. 
Although refinements might improve the structure, they can be confusing to both con- 
sumers and suppliers. 



7. Distribute detailed findings of this project to LEA's as soon as possible. 
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8. Begin a formal pian of action to convey CCEiVrs concept of career educa- 
tion and provide specific direction in terms of required types of commercially produced 
materials. 

In addition, if appropriate, CCEM officials should make available to other 
educational agencies the details of the design and implementation of the various systems 
used in this project. With widespread interest in assessment processes and improved 
management practices, the possible application of the systems and procedures to other 
areas should be analyzed. 

REPORT QVSRVIEW 

This report is organized into six chapters: 
Chapter I - Project Scope, Method and Organization 
Chapter II - Field Survey and Materials Procurement Methods 
Chapter III - Materials Assessment Methods 

Chapter I V - Commercial Career Education Material Information System 
Chapter V - Summary of Survey Results 
Chapter VI - Findings and Recommendations 

Appendixes to this report are presented separately: 

Appendix A - Key Survey Data Print Outs 

Appendix B - Field Survey Procedures Guide 

Appendix C - Assessment Center Staff Manual 

Appendix D - Commercial Career Education Information 
System (CCEMIS) 

Appendix E ^ Summary of Selected Project Management Data 
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With completion of this project, CCEM-I has in its possession narrative descrip- 
tions and documentation, with accompanying flowcharts, of all systems used in the five 
major functional activities; output documents from the information retrieval system; 
abstracts of the 2016 positively assessed materials for entry into the Educational Research 
Information System - ERIC; and recommendations for CCEM-I to continue the project 
using its own staff resources. Files transmitted include hard copy correspondence received 
during this project, identified materials still in process at the completion of this project, 
computer generated reports, and copies of all source documents (completed call reports 
and assessment forms). 



Peat, Marwick, Mitchell & Co. is pleased to have been associated with this pro- 
ject. We believe that the results and the systems used can be of significant assistance in 
furt "^ring the application of the career education concept. 

We wish to express our appreciation to all CCEM-I personnel who assisted in 
the project. They are to be commended and encouraged for their efforts to make public 
school education more meaningful and stimulating. 



Very truly yours, 
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CHAPTER I 

PROJECT SCOPE, METHODS AND ORGANIZATION 



This project was the second major search and assessment efforl conducted 
under the auspices of GGEM-I. The first project, conducted from Novomhcr, 1971 through 
January, 1972, was directed at acquiring and assessing non-coniniorcij/ y produced career 
education materials from such sources as university departments of (education, state 
departments of education, school districts, specially funded curriculum development 
projects, associations, etc. At the conclusion of the firr.t project, it was determined that a 
search and assessment of commercially produced materials was required to provide 
CCEM-I with a complete inventory of currently available career education materials 
appropriate to the Center's curriculum development efforts. 

Thus, the Search and Assessment of Commercial Career Education Materials 
(SACCEM) Project was a nationwide survey of commercially prepared career education 
materials applicable to the actualization of the school-based CCEM. The three following 
project objectives were set forth by contract: 

1. Secure and screen all available commercial educational materials appro- 
priate for a comprehensive career education program as it relates to specific instructional 
units. 

2. Process, assess and classify (index/abstract) selected potential commercial 
educational materials for incorporation into instructional units and/or supportive educa- 
tional activities. 

3. Document activities and submit required reports and information. 

At the outset of the project, ''available commercial education materials" were 
defined as follows: 

Available commercial education materials encompass instruc- 
tional, guidance, administrative, informational support, staff 
development, evaluation and other materials designed to advance 
an educational program in the pupil school environment (grades 
K through 14); that are distributed by any organization which is 
actively engaged in a promotional, marketing and/or training 
program to secure acceptance of a developed proprietary 
product. 

This operational definition was established to provide positive direction for achieving the 
immediate project objectives defined by the CCEM-I RFP No. CEP-2A. 



PROJECT SCOPE 

The project scope is defined in terms of four major project requirements and 
their related major tasks: 

1. Establish a working relationship with commercial organizations that pro- 
vide materials useful in advancing career education to support routine, ongoing dissemina- 
tion of information on available materials. Related major tasks include: 
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Identify major commercial organizations involved in supplying 
desired products to the educational coniniuiiity 

Document trade associations and otb organizations which have 
organized to promote industry ef" ^ss and disseminate 
beneficial data 



Establish procedures for communicating with identified com- 
mercial concerns that provide initial indexing of available ma- 
terials describing the material content and development status 

Provide methods to follow up on contracts, commitments and 
information exchange between the Center and suppliers. 

2. Develop procedures for establishing a data base (using computer process- 
ing services of a Columbus organization) which indexes and classifies available commercial 
materials and their sources. Related major tasks include: 

Develop flexible methods for identifying components of the 
data base and related coding systems which can be implemented 
early in the project and applied consistently over an extended 
period of time. This includes identifying and classifying support 
activity provided by commercial organizations and material 
components 

Design documents which support both survey data collection 
activities and initiation of computer processing 

Document basic computer file structures which support storage 
of information and flexible processing of inquiries 

Establish computer and related manual systems which meet 
both immediate and long-term program needs. 

3. Obtain materials from commercial suppliers and complete required 
screening, classification, assessment, abstracting and indexing of materials commensurate 
with current needs of the CCEM-I program and long-term goals of career education. 
Related major tasks include: 

Work with the Center staff members involved in curriculum 
development to establish short-term, intermediate and long- 
term program needs 

Preview existing materials, including responses from previous 
surveys, bibliographies and other sources to identify products 
from commercial organizations appropriate to the Center's 
short-term priorities 
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Solicit san; materials that screening indicates will meet 
program needs; complete assessment and prepare abstract of 
materials determined to meet established standards 

Procure additional copies of materials judged appropriate as 
required to support Center and Local Education Agency (LEA) 
requirements. 

4. Document survey methods and procedures to support the smooth 
transition from initial project development efforts to routine administration by Center 
personnel. Related major tasks include: 

Provide Center administration with progress reports on perfor- 
mance and budgetary aspects of the project 

Document administrative, coding assessment and abstracting 
activities which must be completed 

Document overall results of initial project, detail recommenda- 
tions for continuing efforts and establish administrative guide- 
lines to assist in managing further survey, classification and 
assessment activities. 

As these requirements and rdated tasks indicate, the project scope was ex- 
tensive. It included not only a national survey of suppliers of educational materials, but 
the design and development of a fully documented process to maintain and update an 
ongoing system for tracking emerging career education materials. 



PROJECT METHODS 

To meet all project objectives within CCEM-Fs 5 month timeframe, a 50- 
member consulting team was assembled from both the educational and business com- 
munities, using national as well as local resources. A project office and assessment center 
was established in Columbus, Ohio. 

The facing page shows the approach used to meet the requirements of a national 
search and assessment of career education materials. (The major project components are 
discussed fully in the following three chapters of this report; the system procedures 
designed and installed to accomplish the subtasks within each component are docur^ent- 
ed in Appendixes B through D.) 

Each of the seven major project components is summarized below: 

1. Identify suppliers . Four man-weeks of desk research resulted in the 
identification of 700 known and potential suppliers of educational materials. Appendix 
B contains a list of 35 sources used for initial supplier identification. 
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2. Secure suppliers' participa tion. Consulting teams based in San Francisco, 
Chicago and New York initiated supplier contacts by telephone and personal visits to 
identify present suppliers of career education materials and to secure their project 
participation. Supplier participation involved: 

Identification and description of proprietary materials via com- 
pletion of product profile sheets 

Agreement to provide complimentary or examination copies for 
assessment 

Permission to index and load a 200 word abstract of positively 
assessed material on the Education Research Information Center's 
(ERIC) system. 

3. Identify career education materials . Completed product profile sheets 
were screened at the project's operation center in Columbus tc ide^ntify materials which 
should be requested from the supplier, 

4. Procure suitable materials . Material deemed appropriate to career educa- 
tion objectives on the basis of criteria established by CCEM was procured through ma- 
terial procurement request to the supplier. 

5- Assess suitable materials. The Assessment Center staff verified that the 
material met the publication/release criteria (between 1965 through 1972). They then 
performed initial classification of the material. Material which passed this screen was 
assessed by professional educators against criteria established by CCEM staff. All first 
level assessments were subject to review by senior assessors. Senior assessors also provided 
final review of global judgments r^-jlated to the material's suitability to CCEM require- 
ments. 

The final step in the material assessment process was a quality control check 
by the project quality control officer using three different methods on a random 
stratified sampling basis. Material which successfully met all existing criteria and passed 
the quality control check was then determined to be "Positively Assessed Material" 
(PAM) and was routed for indexing and abstracting. 

6. Index and abstract. Positively assessed material was indexed (per ERIC) 
and a maximum 200- word abstract was written using basic ERIC descriptions. Upon 
completion, PAMs were prepared for delivery to ERIC staff. 

7. Deliver materials and instruments . A finnl check of the required instru- 
mentation was performed on PAMs. Instruments accompanied by complimentary copies 
were routed to the ERIC staff. Where the supplier had provided examination copies, the 
instruments only were routed to ERIC, and the examination copies were returned to the 
supplier. The ERIC staff performed the quality control check on the appropriateness of 
the index/abstract document. 
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Number in parenthesis indicates the number of professional staff within each function. 
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Total documentation was maintained throughout this process. Nearly 90 
percent of the total documentation was inputed as generated to the computer based in- 
formation retrieval system designed specifically for this project. Actual computer svstf> 
maintenance was performed by a local service bureau. A basic system Mlljpr|^^ , 
version to client operation. 



PRDJECT ORGANIZATION 

To implement tihe siurvey method, the p^-oject team efforts were organized into 
5 system components: 

Field Survey Activities 

Assessment Center Operations 

Quality Control Procedures 

Systems Development (Data Retrieval ) 

Support and Administration. 

The facing page shows the project staff organization. The Project Directci: had 
the assistance and counsel of selected PMM&Co. partners and outside legal counsel. The 
PMM&Co. partners served in an advisory capacity and' provided quality checks on the 
project by participating in milestone report meetings. The outside legal counsel reviewed 
copyright considerations in indexing and abstracting assessed materials and provided gen- 
eral legal counsel. The following paragraphs briefly describe the major subtasks of each 
function: 

1. Field Survey with responsibility for: 

Identifying commercial suppliers 

Identifying career education materials 

Conducting the national survey contacts 

Performing initial classification and screening of 
identified materials 

Initiating material procurement requests for materials 
apparently directed at career education. 

2. Assessment center w-ith responsibility for : 
Performing materials assessment 
Indexing and abstracting materials 
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PerformiiiR final assossmoni review 

Maintaining material control procedures Tor entry and 
exit of materials 

Delivering materials to ERIC 

Verifying data for input to CCEMIS. 

3. Qu ality control with responsib ility tor : 

Consulting on assessment procedures design 

Initial staff development and training for 
assessiBent personnel 

Assisting in designing and developing Assessment 
Manual 

Conducting ongoing quality control checks of assessed 
materials 

Specifying qujility control remedial tasks, 

4. Sys tems deve lo pments with responsibility for : 

Documenting data base requirements 

Initial development and procedure specifications to 
support project activities 

Designing source forms 

Designing general system 

Selecting outside service bureau 

Conducting ongoing review of service bureau per- 
formance. 

5. Support and administration with responsibility for: 

Task/expense reporting on a two-week basis 

Maintaining all project expenditure records 

Assisting in budget forecasting, revision and 
management 
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Providing project management data in compliance with 
terms of contract. 

The project components, major tasks and functional activities are indicated on 
the project schedules shown on the following pages. The project was completed as sched- 
uled, although portions of the schedule were revised within the Phase II timeframe 
(see Appendix E). 

One final measure of the project scope is that more than five man-years (more 
than 10,000 hours) of effort were applied within the 4-month project life. 
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CHAPTER II 



FIELD SURVEY AND MATERIALS PROCUREMENT METHODS 



This chapter presents our general approach and describes the major field survey 
tasks. The details on field survey procedures, forms documentation, and resources are 
contained in Appendix B. 



SURVEY APPROACH 

The primary objective of the survey was to achieve a large number of appro- 
priate source contacts which would generate completed product profile sheets so that the 
materials procurement process could begin with a significant flow of materials. We 
attempted to achieve this goal with minimum procurement of inappropriate materials to 
maximize the time and efforts expended at the Assessment Center in screening and eval- 
uating materials. 

The initial sourcing effort included a thorough search of the following: 

Files and other resources of the CVTE 

Information systems such as ERIC and AIM/ ARM 

Products of recent research in your career education projects 

Files and data generated during the first CCEM assessment project 

Files and publications of associations most relevant to the producers 
of career education materials 

Commercial indexes and catalogs 

Recognized experts in the field. 

The intended results of the research effort included: 

1. The generation of an extensive and comprehensive inventory of commer^ 
eial firms most likely engaged in the production of career education materials. 

2. The identification of businessmen and professional personnel who would 
be likely sources of relevant data and source information. 

3. The identification of career education materials suitable for initial pro- 
curement efforts. 

The field survey effort, which included telephone, personal and mail contacts with 
identified sources and/or potential sources, was directed toward achieving these results. 

The field survey efforts were centrally directed from Columbus, Ohio but were 
organized and conducted on a regional basis in New. York, Chicago and San Francisco 
Teams were assembled in each of these cities to initiate contacts and perform the neces- 
sary foUowup with sources in their regions. The teams were staffed with professionals who 
had both the training and experience necessary to rapidly gain the cooperation and 
participation of commercial suppliers. 
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An initial test of suppliers' cooperation was conducted during the first week of 
field effort among 25 to 30 sources located in New York and Chicago. The results of the 
field test verified the survey approach and provided insiglits for modifying the procedures 
for use in the full-scale survey efforts. 



FIELD SURVEY TASKS 

The field survey function was responsible for procuring complimentary copies 
of examination copies of materials from commercial suppliers. The major tasks involved 
were: 

Identifying suppUers 

Securing suppHers' participation 

Initiating product profile sheets on identified materials 
Following up to faciUtate material procurement. 

The facing page shows the procedures and steps used to procure materials for 
assessment. 

The following pages show the Call Report and Product Profile Sheet used to 
document the initial steps of the procurement process. These are two of the four basic 
input documents to the information retrieval system. 

Before the full scope implementation of the supplier contact program, a 1-day 
training session was conducted for each of the three regional teams. The training session 
included: 

Project overview 

Description of the total procurement and assessment process 
and general search objectives 

Suggested telephone procedures and criteria for onsite visits to 
sources 

Suggested approaches for securing source cooperation 

Detailed review of field survey documentation requirements 
including call reports, product profile sheets, material 
acquisition requests 

Review of daily phone contact procedures with the Assessment 
Center. 

The initial telephone contact with suppUers was followed with a letter (shown 
on the following pages) which formally requested the suppliers' participation. Appendix 
D contains the telephone procedure guide and all form letters used during the survey; a 

O . detailed summary of the desk research conducted to identify suppliers; and a Usting of 

JC the resources used for this purpose. 
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Peat. Mahavtck. Mitchell & Co, 

CEHXmED PUBl-IC ACCOTTKTANTS 
222 SOfTH Rrv^ERSIDE PLAZA 
CHICAOO. ILUKOIS BO(306 



August, 1972 



Dear Sir: 

Peat, Marwick, Mitchell & Co. (PMM&Co.) has been designated as subcontractor 
to the Center for Vocational and Technical Education: The Ohio State University by the 
USOE to conduct a national survey and assessment of selected career education materials 
commercially available. 

In summary, PMM&Co. has been engaged to identify, procure and assess com- 
morciaily available career education materials against criteria established by the Compre> 
rrcnsive Cateer Education Model (CCEM) which has been created by USOE and the Center. 
Over the ninety-day period of the contract, PMM&Co. is to determine the suitability of 
available career education materials for grades K-14 for the career development of each 
student which: 

Restructures the entire educational program around the student's 
real-life developmental needs 

Integrates the academic knowledge and skills with occupational 
training 

Assumes that each existing student will be prepared for a future 
career education program and for entry into an occupation 

Provides for each student a program relevant to his becoming a 
self-fulfilled productive and contributing citizen 

Incorporates community resources and non-school educational 
opportunities. 

Materials which meet CCEM and career education needs will be abstracted, 
indexed and entered into the national information storage, retrieval and dissemination 
system for vocational-technical education through the Vocational Technical-Education 
Resource Information Center (VT-ERIC) clearing house. 

To complete this task, PMM&Co. has been in telephone contact with companies 
and industries throughout the country which are serving the career education market. 
Based on our findings, we have learned that your firm has materials which hopefully will 
Q be made available for our processing. 

ERIC 



August, 1972 
2 



Specifically, we request: 

(1) Completion ot a product profile sheet for each piece ot ca- 
reer education material which you market. 

(2) Agreement to provide copies of materials which we request 
after review of the product profile sheets. 

With authorization of CCEM and USOE officials, we plan to make available at 
the conclusion of the project a listing of: 

Companies serving the career education market 

The number of career education materials identified by grade 
level and career education element 

A summary of assessed materials by degree of suitability to 
CCEM requirements by grade level and career education 
elements. 

Of course, no identification will be made of materials b> dtle or source. 

We appreciate the preliminary indication of the willingness of your company to 
participate in the project. Your cooperation and assistance in having the attached product 
profile sheets completed and returned to us at the earliest possible date will be welcomed. 

If you have questions regarding our request, please contact: 

Dr. Voie Coy 
41 South High Street 
Columbus, Ohio 43215 
(614) 221-7258 



Thank you for your help. 



Very truly yours, 



PEAT, MARWICK, MITCHELL & CO. 



Clyde N. Carter 
Principal 
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CHAPTER III 
MATERIALS ASSESSMENT METHODS 



The chapter briefly describes the materials assessment methods used in process 
v:a\\ areer it>ducation materials. Specifically, it presents an overview of the process, the 
<i:.ssessme:u instrument, method of assessing, and quality control procedures. Appendix C 
presents a full explanation of all aspects of assessment. 



THE ASSESSMENT PROCESS 

The Assessment Center revolved around two interrelated functions: 

1. The flow of material from entry into the system, through the priority 
> ens and direct assessment, on through exiting and shipment to the client. 

2. The records control of all material and entry of data into the information 
retrieval system. 

The procedural and sequential flow of Assessment Center activities is shown on 
the facing page (refer to steps in the process). This schematic does not represent the 
physical movement of an item. The organization of the Assessment Center as a total 
process and the actual work and record flow are shown on the following pages. The work- 
flow for each record or piece of material and the relationships of work stations are also 
indicated. 

The assessment process included the following conducted in logical sequence: 

1. The field survey team contacted suppliers and asked for materials which 
matched the Product Profile Sheet. 

2. The completed Product Profile Sheets were reviewed at the Assessment 
Center to determine if the materials would be procured and if they were of "immediate- 
access" priority or could be postponed until later in the project. 

3. Materials received at the Assessment Center which appeared to meet CCEM 
priority requirements were entered into the assessment process immediately or held in 
storage ( 'Hold") until a later time because of low immediate priority (need determined 
by the client). 

4. Material entered into the assessment system which had been identified and 
classified according to the CCEM matrix arrived at a "Go/No-Go" point, (i.e., it matched 
or did not match the matrix requirements): 

Items which did not match were exited from further assessment 

Items which matched were assessed fully unless they were put 
in a "Hold" category. 

5. Each item was assessed for its overall suitability and applicability to the 
project requriements. 
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Thus, the assessment process consisted of screens or judgment decision points 
about each item: ''Does the item meet the client requirement?*' and 'is the item needed 
by the client now or can it be delivered later?*' 



THE ASSESSMENT INSTRUMENT 

The assessment instrument is shown on the following pages. Its content was 
determined on the basis of client need and a quality level of educational evaluation. The 
format of the instrument was dictated by assessment procedural flow requirements, the 
psychological set of the assessors as they worked through the problem, and the data 
system requirements. The format was also affected by the need to ensure quality by re- 
ducing error possibility (e.g. recopying marks) and by the procedural and physical flow of 
materials. The design process was a joint effort of CCEM staff and the project consultants. 

The assessment instrument was labeled, numbered and attached to the curric- 
ulum materials to be assessed. Two official numbers on the instrument made up the 
Document Number: the Call Report Number and the Product Profile Sheet Number. The 
Product Profile Sheet number was used to keep track of the materials and assessments 
instruments while they were in the Assessment Center. 

Materials were assessed by assessors functioning in their areas of expertise. 
Senior assessors made the summary (global) judgments and evaluated the assessors judg- 
ments. Other specialists and technicians performed specialized assessment functions which 
did not occur routinely (e.g., 5 D.O.T. classifications). 



INDEXING AND ABSTRACTING 

Material which passed all of the screens was judged positively assessed. Positive- 
ly assessed material was processed for indexing and abstracting as prescribed by ERIC 
indices and descriptors in preparing abstracts of up to 200 words. Materials and their 
instrumentation were then routed to the ERIC staff for inclusion in the ERIC system. 

Clerical and administrative technicians handled records management, item 
coding and entry and exit processes for materials. 



QUALITY CONTROL 

An integral part of the assessment process was the maintenance of uniform 
standards of consistency and accuracy among assessors and within the performance of an 
individual assessor. The quality control function was the assigned responsibility of a 
senior professional designated as the Quality Control Officer. The Quality Control Officer 
actively participated in: 

Project design 

Assessor development and training 

Design and development of the Assessment Center Staff Manual 
Maintenance of quality performance during the ongoing 
assessment process. 
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SUPPLEMENT FOR X-2J MATERIALS 



For items which fit the following target areas: 
E. Staff development 
H. Support systems 
L Community relations 
L. Management 

Check applicable items: 



1. 


Elementary level 


2. 


Secondary level 


3. 


Post-secondary level 


4. 


Staff-development/teaching methods 


5. 


Staff-development/career education programs 


6. 


Staff-devf/Iopment/vocational education programs 


7. 


Staff-development/counseling methods 


8. 


Staff-development/materials preparation 


9. 


Staff-development/other instructional staff 


10. 


Administration/career education programs 


11. 


Administration/vocational education programs 


12. 


Promotional/career education; community focus 


13. 


School-community communication 


14. 


Other 



Suitability to CCEM requirement: HAL 



Additional Comments: 



ASSESSMENT INSTRUMENT « SUPPLEMENT 



This form to be used only for items on page 5 of the Assessment Instrument as listed 
below. 

SPECIAL CONDITIONS FOR USER 
EQUIPMENT & FACILITIES 

If this item is checked, indicate below what special facilities, equipment 
or transportation is needed. 

Special facilities : 

Special equipment : 

Special transportation: 

PERSONNEL AND STAFFING 

If this item is checked, indicate below what kind of staffing or specially 
trained personnel is required. 

Differentiated staffing : 

Special support personnel : 

Specially trained instructor : 

SCHEDULING & ORGANIZATION 

If this item is checked, indicate the nature of the grade organization 
required and the nature of the time requirement outside the normal 
day or year. 

Unique grade organization : 

Outside normal day/year : 



111-3 



The Assessment Center Director and Supervisor perfoimed dailv quality con- 
trol checks through over-the-shoulder coaching, retraining of assessors as needed and 
answering questions when assessors faced interpretation problems. In addition, Senior 
Assessors (professionals with considerable experience in instructional materials develop- 
ment and curriculum and program development) provided judgments on a material's 
overall suitability to CCEM requirements. Finally, the Quality Control Officer performed 
three types of procedural checks of completed assessment documents. 

1. Total Chgck. At least 10 percent of the materials which had been assessed 
were set aside. The Quality Control Officer used the form, shown on the following page 
to make an mdependent reassessment and to verify the completeness and legibility of 
work, interpretation of every item, correctness of the decision and the internal consist- 
ency between interrelated items. Documents with errors were returned to the Supervisor. 

2. BHnd Check. A weekly sample of materials which had exited were restart- 
ed through the assessment procedure by a second asse.ssor. The completed assessment 
mstruments of the two assessors were compared by the Quality Control Officer to deter- 
mine if the interpretation and judgment of the two assessors was consistent. 

3. Spot-Check. During on-site visits, the Quality Control Officer sampled 
completed items to spot-check certain crucial items (e.g., exit decisions, career element 
consistency between matrix elements). 



In total, more than 16 percent of the assessment instruments were subjected 
to a quality control check by the Quality Control Officer. 



Assessors Work Check I 



Date 



1. Name^ 



Document # 



2. Type of Item 

3. Probable Level of Assessment Difficulty? H 



4. Exit Point of Material 

5. Type of Check Made: 

— a. Reassess blind check, _ 

6. Checked By_ 

7. Action Taken: 

a. Accepted and filed 

b. Corrected and accepted 



X-l 



X-2 



X-3 



b. Spot-check selected items, 



c. Total cks. 



8. Items Checked: 

a. Completeness 

c. Consistency Between Items: 

- (1) 33 & 20-23 



c. Rejected: 

Redone with assessor 



Assigned to 

supervisor 

Verified 



b. Legibility 



- (2) 32& 19 



(3) 31 & 11/17 



- (4) 29 & 1245 

^ (5) 



d. Accurate Definition of Terms - Errors Noted 

(1) 

(2) 

(3) 

(4) 



e. Exit Decision Correct? Yes. 

f. Other Questions: 



No. 



EKLC 



Note: Record on weekly log & file by assessor name. 



IV - COMMERCIAL CAREER EDUCATION MATERIAL INFORMATION SYSTEM 



ERIC 



CHAPTER IV 



COMMERCIAL CAREER EDUCATION MATERIALS 
INFORMATION SYSTEM (CCEMIS) 



The Commercial Career Education Materials Information System (CCEMIS) is 
a computerized system for storing and retrieving descriptive and evaluative data concern- 
ing career education materials. The system accepts all data recorded for each item iden- 
tified as career education material. The retrieval system component can seek out and 
print all documents relevant to a particular sort descriptor or combination of descriptors 
such as Grade Level, Target Area, CCEM Element/Target Area, etc. 

CCEMIS accepts descriptors of career education materials from input forms 
for data storage and retrieval. The facing page shows the overall operation of the system. 
Data is entered into the system from three basic forms: Call Report, Product Profile 
Sheet and the Assessment Instrument. A fourth form, the Change Form, is used to modify 
or add data to existing documents (Appendix D). 

The input data creates tv^o files: 

1. Call Report File is maintained as a tape file and used only for the report or 
organizations contacted (Appendix D). It has one entry for each company or corporate 
division to which a call was placed. 

2. Document File Maste r contains all information from the Product Profile 
Sheets and Assessment Instrument* All entries on the file are referred to as documents 
and are identified with nine digit document numbers. The first four numbers are the Call 
Report number, unique to each company, and the last f've numbers are the preprinted 
number from the Product Profile Sheet. 

A document is first entered in the data file when a Product Profile Sheet enters 
the system with one of the following General Status Codes: 

A - Product Profile Present, 
No Material Requested. 

B - Product Profile Present, 
Material Requested. 

The material from the Product Profile Sheet is entered into the file and space 
is left for adding all information contained in the Assessment Instrument. When the mate- 
rial is assessed and enters the system, the information from the Assessment Instrument is 
entered into the system and the General Status Code is automatically changed as follows: 

C - Assessed, Exits System at Exit 1 
D - Assessed, Exits System at Exit 2 
E - Assessed, Exits System at Exit 2 

Under Condition ''J." 
F - Fully Assessed, Exits System at Exit 3. 

The system produces six reports, examples of which are reproduced in Exhibit 1: 

1. Edit, Update and Change List . Prepared each time new information is 
added to the file, primarily for system control. 

EKLC 



1V.2 



2. Organizations Contacted. Lists all organizations contacted and the product 
profiles submitted by the organization. 

3. DetailJFjle^^ Lists all information on the document file for backup 

purposes. 

4. Title-Author Report. Lists the Document Number, Title and Author of 
all documents on the file. 

5. StatisticaLReport. Provides statistical information about the total items 

in the file. 

6. Material Selection Report . Provides the information needed by an edu- 
cator for a specific application for career education material. The user can specify all the 
conditions of his environment and requirements and CCEMIS will retrieve and list all 
documents on the file v^rhich meet his specific needs. 

The specific procedures for using CCEMIS are documented in Exhibit 2. The 
system has been designed to provide maximum information with minimum user effort. 



CCEMIS Reports 
(Illustrative Only) 



Edit, Update and Change List (Call Report) 

Organizations Contacted 

Detail File Report 

Title - Author Report 

Statistical Report 

Material Selection Report 
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CALL NO. 
Description 
Pub Year 
Rev Year 
Copy Year 
65 or Later 
Ind/AbstrcCt? 
Gen Status 
Mat'l Status 
Title 



Author 



Single Item? 

Item Cost 
Set? 

Cost/Set 

Pieces/Set 
Separate? 

Cost 1 

Cost 2 

Cost 3 

Cost 4 

Cost 5 

Cost 6 
Type Mat'l 
Target Area 
Content/Disc 

Population 
Grade Level 



PPS NO. 
Field Value 



COl 
C02 
C03 
C04 
COB 
C06 
C07 
COS 



CIO 
Cll 
C12 
CI 3 
C14 
C15 
CSl 
CS2 
CSS 
CS4 
CSS 
CS6 
C17 
C18 
C19 

C20 
C21 



K-3 Focus 
4-6 Focus 
7-14 Focus 

DOT Codes 



CCEM I - DATA CHANGE FORM 



De_scr_iption 
Element ^1 
-1 Grade 



DATE 



Field Value 
C45 
D45 



2 


3 
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6 7 8 
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Mm 
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A B : u E F G 
1 2 3 4 5 6 



C09 I 



_L 



Element p2 
"1 Grade 



Element #3 
-3 Grade 



C46 
D46 



1 2 3 4 5 6 7 8 



J/ 
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9 
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C47 
047 



1 2 3 4 5 6 7 8 
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a 1 

























B- 
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E 













Exit X-2? 
Exit Reason 

Element #1 



C48 Y 
C49 J K L M N 

C50 12 3 4 5 6 7 8 



1 1 1 1 1 -1 Taraet D50 
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C 
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MM! 
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Element n C51 1 2 3 4 5 6 7 8 
-1 Target D51 



A B C D E F 

ABCnEFGHIJKLM 
ABCDEF6HIJKLM 
N 0 P Q R S T U 
1 2 3 4 5 6 

Kl 23456789ABC 
D E 
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F 


G 
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M 















Element #3 
-3 Target 



C52 
D52 



1 2 3 4 5 6 7 8 
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G 

















Packaging 


C22 
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i. 
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4 




Printed Mafl 


C23 
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E 
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P 
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Films 


C26 
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Manipulative 


C27 
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Au'dio-Visual 
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Cost Factors 
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Equip/Facil C58 

Pers/Staffing C59 

Sched/Org C60 

Comm Resources C61 

Pers Factors C65 

Spec Learn C69 

Type C70 

Inst Strat C62 



C37 A B 
C39 1 2 3 4 5 
C42 ABCUVWXFGYZIJ 
K L M 

C44 m m 



Time 

Descriptors 
Judgment 
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*** DETAILED FILE REPORT *♦* 



PAGE 49 



1513-07007 


BE INFORMED ON FINDING A JOB 


BAILEY ic CHIDEYA 


PUB YR 


68 


SIN CC'ST 


TYP 




REV YR 


68 


SET COST Y 1 


TA 


A 


C/R YR 


68 


SET Q TY 5 


CD 


GP 


PUB t5 


Y 


SEP PUR CSTl 


POP 


1 


ABSTRACT 


Y 


2 


OR 


3 


GEN STAT 


D 


3 






MR STAT 


2 


•a 






PACK 


1 


K- 3 






PMTL 


P 


4-6 






FILM 




7-9/10-14 




MMTL 










AMTL 










COST 


1 


DOT 






EX 1&2 Y 


L 








CCEM/GR- 


ELEM 




CCEM/TA ELEM 




ELEM 






ELEM 




ELEM 






ELEM 



•** DETAILED FILE REPORT *** 
11/06/72 PAGE 50 



1513-07008 




BE INFORMED READING YOUR NEWS 


STAUFFER 


PUB YR 


68 


SIN COST 




TYP 


A 


REV YR 


68 


SET COST Y 




TA 


A 


C/R YR 


68 


SET QTY 


5 


CD 


ADQ 


PUB 65 


Y 


SEP PUR CSTl 




POP 


1 


ABSTRACT 


Y 


2 




GR 


34 


GEN STAT 


A 


3 








M R STAT 


2 


4 








PACK 












PMTL Y 






4-6 






FILM 






7-9/10-14 




MMTL 












AMTL 












COST 6 






DOT 






EX 1&2 













CCEM/GR- ELEM 
ELEM 
ELEM 



CCEM/TA ELEM 
ELEM 
ELEM 
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*** STATISTICS REPORT *** 



2 



GENERAL NUMBER NUMBER COST NUMBER TYPE 



STATUS 


DOCUMENTS 


PIECES 


CODE 




RAIL 


A 


841 


11858 


1 


2388 


Printed 2278 


B 


422 


4650 


2 


454 


Film 1362 


c 


200 


1003 


3 


117 


iVldnip X I U 


D 


128 


1030 


4 


14 


Aud Vis 447 


E 


77 


188 


5 


69 


F 


1939 


14885 


6 


565 




TOTAL A-F 


3607 


33614 


TOTAL 


3607 





G 



GRADE 


NO. 


EXIT 


NO. 


INST, 


K 


258 


A 


1 


A 


1 


338 


B 


124 


B 


2 


391 


C 


2 


C 


3 


395 


D 


23 


D 


4 


412 


E 


84 


E 


5 


437 


F 




F 


6 


544 


G 




G 


7 


975 


H 




H 


8 


1096 


I 




I 


9 


1414 


J 


78 


J 


A 


1482 


K 


13 


K 


B 


1708 


L 


99 


L 


C 


1749 


M 


9 


M 


D 


824 


N 


22 


N 


E 


836 






0 



p 

Q 
R 
S 
T 
U 
V 
W 



NO. 


TIME 


NO 




Mn 


1357 


A 


921 


A 


K 
v 


1709 


B 


1 


B 


ou 


1 


C 


1 


c 


51 

W J. 


31 


D 


12 


D 


6 


23 


E 


3 


E 


132 


47 


F 


4 


F 


506 


1 


G 


8 


G 


110 


127 


H 




H 


454 


99 


I 




I 


374 


86 


J 




J 


21 


178 


K 




K 


546 


1237 


L 


982 


L 


5 


24 






M 


6 


83 






N 


649 


1018 






0 


33 


11 






P 


55 








Q 




26 






R 


166 


100 






S 


27 


23 






T 


2 


16 






U 


253 


36 






V 


221 



11/07/72 *** STATISTICS REPORT 

CCEM MATRIX .\ND GRADE LEVELS 
K 1 

SELF AWARENESS 
H 3 7 

A 8 9 

L 2 2 

X 



2 


3 


4 


5 


6 


7 


8 


9 


10 


n 


12 


13 


14 


10 


10 


4 


5 


7 


7 


8 


11 


20 


36 


36 


11 


11 


16 


11 


8 


7 


9 


28 


33 


47 


54 


59 


60 


20 


19 


3 


5 


4 


7 


16 


20 


25 


36 


32 


38 


32 


5 


5 



EDUCATIONAL AWARENESS 



H 






1 


1 


1 


1 


2 


29 


41 


51 


55 


72 


74 


18 


A 


5 


5 


8 


5 


14 


16 


30 


78 


91 


128 


136 


186 


181 


58 


L 


6 


12 


ry 

1 


14 


31 


39 


61 


97 


101 


168 


162 


191 


189 


73 



CAREER AWARENESS 
H 38 19 27 23 

A 127 171 188 126 

L 30 42 62 89 



21 40 45 63 76 
86 96 129 403 430 
142 142 170 160 169 



117 149 236 236 43 42 
547 603 680 666 195 193 
191 113 101 93 42 42 



ECONOMIC AWARENESS 



1 


1 


5 


5 


7 


6 


6 


4 


5 


7 


13 


12 


12 


2 


2 


5 


7 


11 


11 


10 


7 


8 


7 


14 


17 


30 


57 


58 


14 


14 


2 


3 


7 


8 


7 


11 


13 


10 


15 


20 


23 


22 


23 


3 


3 



DECISION MAKING 

H 1122221333444 

A 1112223459 13 14 55 

L 1 4 4 5 4 5 4 

X 



BEGINNING COMPETENCY 
H 

A 12 12 12 12 

L 1 
X 

EMPLOYABILITY SKILLS 

H 

A 

L 

X 















7 


20 


20 


11 


11 


1 


1 


1 


7 


7 


12 


36 


67 


63 


30 


29 


1 


2 


2 


5 


5 


26 


35 


39 


33 


16 


16 






1 


1 


1 


3 


12 


25 


25 


9 


9 


1 


1 


1 


16 


21 


28 


43 


61 


60 


27 


27 






1 


5 


8 


22 


25 


46 


45 


23 


23 



ATTITUDES & APPRECIATIONS 

H 6 10 15 13 8 23 23 32 21 26 39 46 46 7 7 

A 52 96 125 148 141 147 200 229 253 274 207 222 207 74 73 

L 71 81 92 45 56 62 83 103 117 125 151 159 151 15 15 



11/07/72 *** STATISTICS REPORT *** 

CCEM MATRIX AND TARGET AREAS 

ABCDEFGHI J KLI^! 
SELF AWARENESS 

H 11 38 12 38 1 

A 25 66 29 44 1 2 1 

L 9 35 27 16 1 

EDUCATIONAL AWARENESS 
H 3 94 11 54 

A 16 246 31 116 1 

L 15 243 11 101 1 

CAREER AWARENESS 

H 35 505 37 138 1 

A 91 712 93 213 2 2 1 1 4 

L 93 336 37 73 1 

ECONOMIC AWARENESS 
H 15 13 4 5 1 

A 18 69 12 26 1 

L 12 41 4 3 

DECISION MAKING 

H 3 3 2 3 2 

A 8 5 7 8 1 

L 12 1 

BEGINNING COMPETENCY 
H 7 10 26 1 

A 36 39 53 14 5 

L 2 33 33 5 2 2 

EMPLOYABTLITY SKILLS 
H 2 9 24 6 

A 7 53 29 42 1 ] 

L 2 32 18 27 1 

ATTITUDES & APPRECIATIONS 
H 18 85 12 30 

A 117 505 70 141 11 11 

L 24 293 30 24 
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11 07 72 *** STATISTICS REPORT 

JUDGMENT 



OCCUPATIONAL FOCUS 
INTERNAL CONSISTENCY 
CCExM MATRIX BY GRADE 
CCEM MATRIX BY TARGET AR 
NEED FOR SPEC. CONDITION 
SELi -COM. PLUG-IN UNIT 
INNOVATIVENESS, SUBSTANTI 
INNOVATIVENESS, PR-OCEDURA 
PARTICIPANT INVOLVEMENT 



HAL 

781 690 466 

208 1686 44 

180 1388 370 

505 953 ASn 

• d 1928 

71 434 1432 

37 304 1596 

162 452 1324 
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CCEMIS 



FIELD DEFINITION CODES FOR 
MATERIAL SELECT PROFILE 



CCEMIS 



FIELD DEFINITION CODES FOR 
MATERIAL SELECT PROFILE 



Document Number 

General Status Code 

A. Product Profile Only 

B. Materials Requested 

C. Assessed - Exit 1 

D. Assessed - Exit 2 

E. Exit 2 Type J, Indexed & Abstracted 

F. Fully Assessed, Indexed & Abstracted 

G. Inactive Product Px-ofile 

(Items assessed individually) 

Materials Request Status (from Call Report) 

1. Complimentary Copy 

2. 2 Complimentary Copies 

3. Loan (Examination) 

4. Rental 

5. Purchase 

6. Undecided 

(S) Set Y- 

Target Area 
A. Curriculum 

Career Information 

C. Career Preparation 

D. Guidance 

E. Staff Development 

F. Placement/Guidance 

G. Evaluation 

H. Support Systems 

L Community Relations 

J. Post Secondary 13, 14 

K. Rural Education 

L. Management 

M. Pe:.;3onnel Services 

Content/Discipline Area Breakdown 

A. Language Arts 

B. Mathematics 

C. Sciences 

D. Social Sciences 

E. Fine Arts 

F. Health/Physical Education 

G. Career Education 

H. Agriculture/Natural Resources 

I. Office Education 

J. Distributive Education 

K. Health Occupations 

L. Home Economics Education 



FIELD DEFINITION CODES (COm. \ 



M. 


Industrial Education 


N. 


Consumer Education 


0. 


Industrial Arts 


P. 


Vocational Education 


Q. 


Business Education 


R. 


Data Processing 


S. 


Recreation 


T. 


Not Used 


U. 


Technical Education 


V. 


Other 



Career Education Population Served 

1. Students 

2. Instructional Personnel 

3. Non-Instructional Personnel 

4. Parents 

5. Community at Large 

6. Other 

Grade Level 
K 
1 
2 
3 
4 
5 
6 
7 
8 
9 
A 
B 
C 
D 
E 



Printed Materials 

A. Instructional Guide/Manual 

B. Teaching Guide/Manual 

C. Curriculum Guide 

D. Program Planning Guide 

E. Unit of Study (subject fields) 

F. Instructional Module 

G. Student Workbook 

H. Student Manual 

I. Laboratory/Practice Manual 
J. Textbook/Workbook 

K. Simulation/Environmental Situation 

L. Evaluation Instruments 

M. Administrative Materials 



FIELD DEFINITION CODES fCONT>) 



N. Bibliographies 

0. Directories 

P. Supplementary Reference Materials 

R. Textbook 

S. Other 

Film 

1. Films 

2. Filmstrips 

3. Cassettes 

4. Videotapes 

5. Slides 

Manipulative Materials 

A. Puzzles 

B. Laboratory Kits 

C. Puppets/Dolls/Figures 

D. Activity Kits 

E. Tools & Related Items 

F. Games 

G. Blocks & Related Items 

H. Rods/Counters/etc. 
L Other 



Audio- Visual Materials 


1. 


Records, Discs 


2. 


Audio Tapes 


3. 


Transparencies 


4. 


Models/Mock-ups 


5. 


Flat Pictures 


6. 


Posters 


7. 


Charts & Graphs 


8. 


Single Concept Loops 


9. 


Other 


Cost Factors 


1. 


Less than $100 


2. 


$101 - $250 


3. 


$251 -$500 . 


4. 


$501 - $750 


5. 


Over $750 


6. 


Unknown 



Exit X-1 Y- 



Exit Conditions 
(a) Exitl 

A. Cost 

B. Textbook 

C. Non-Commercial Material 

D. Incomplete Component 

E. Other 



FIELD DEFINITION CODES (CONT. ) 



C37 K-3 Focus 

A. Good 

B. . ervices 

C39 Grades 4 • 6 



C42 



1. 


Industry 


2. 


Commerce 


3. 


Social Sciences 


4. 


Services 


5. 


Arts 


Grades 7 - 14 


A. 


Natural Resources 


B. 


Construction 


C, 


Manufacturing (u) 


D. 


Transportation & Communication (v) 


E. 


Trade and Finance (w) (x) 


F. 


Government 


G. 


Education 


H. 


Health & Welfare (y) (z) 


I. 


Personal Services 


J. 


Product Services 


K. 


Arts & Humanities 


L. 


Recreation & Entertainment 


M. 


All of the Above 



C44 DOT Occupational Group Code (5) 

C45,C46, Element Code 

C47 CCEIVI Matrix and Grade Level 

(a) Element Code 

1. Self Awareness 

2. Educational Awareness 

3. Career Awareness 

4. Economic Awareness 

5. Decision Making 

6. Beginning Competency 

7. Employability Skills 

8. Attitudes and Appreciations 

(b) Grade Level 

Rating 
L: Low 
A: Average 
H: High 

X: Relevant but not assessed 
Exit X-2 Y. 



Matrix 
Elements 



D45,D46, 
D47 



C48 



^Letters in parentheses appear only on Detailed File Report. 



FIELD DEFINITION CODES (COm. ) 



C50,C51, CCEM Matrix and Target Areas 
C52 (a) Element Code 

1. Self Awareness 

2. Education Awareness 

3. Career Awareness 

4. Economic Awareness 

5. Decision Making 

6. Beginning Competency 

7. Employability Skills 

8. Attitudes and Appreciations 



(b) Target Areas 

A. Curriculum 

B. Career Information 
G. Career Preparation 

D. Guidance 

E. Staff Development 

F. Placement 

G. Evaluation 

H. Support Systems 

I. Community Relations 
J. Post Secondary 13, 14 
K. Rural Education 

L. Management 

M. Personnel Services 



Matrix 
Elements 



D50,D51, Rating 
D52 L: Low 



A: Average I Matrix 

H: High ( Elements 

X. Relevant but not assfi>sed 



SPECIAL CONDITIONS FOR USER 



C58 Equipment & Facilities 

1. Open Space, Multiple Group 
Instruction 

2. Special Facilities 

3. Special Equipment 

4. Special Transportation 

5. Other 

Personnel and Staffing 

A. Team Teaching 

B. Differentiated Staffing 

C. Low Pupil-Teacher Ratio 

D. Paraprofessional support 

E. Peer Student Support 

F. Special Support Personnel 

G. Specially Trained Instructor 
H nf.hpr 



C59 



ERLC 



FIELD DEFINITION CODES (COmA 



Scheduling & Organization 

1. Unique Grade Organization 

2. Outside Normal Day/Year 

3. Modular or Flexible Schedule 

4. Community Organization Based Classroom 

5. Industry Based Classroom 

6. Business Based Classroom 

7. Other 

Community Resources 

A. Volunteer Support, General or Specific 

B. Industry/Business Instructors 



c. 


Community Agency Support 


D. 


Work Experience Stations 


E. 


Other 


Instructional Strategies 


A. 


Self-Instruction 


B. 


Discussion 


C. 


Micro-Teaching 


D. 


Programmed Instruction 


E. 


Modeling 


F. 


Simulation or Gaming 


G. 


Psychodrama 


H. 


Individualized Projects 


I. 


Role Playing 


J. 


Field Observations 


K. 


Self-Evaluation 


L. 


Group or Panel Discussions 


M. 


Demonstrations 


N. 


Drill and Practice 


0. 


Individual or Group Counseling 


P. 


Supervised Work Experience 


R. 


Lab Simulation/Practice 


S. 


Problem Solving 


T. 


Dramatic Activities 


U. 


Manipulative Skill Development 


V. 


Psychomotor Activities 


w. 


Other 


Time Needed 


A. 


1 - 5 (clock hours) 


B. 


6-10 


C. 


11-20 


D. 


21-30 


E. 


31-45 


F. 


90 (one semester) 


G. 


180 (tv/o semesters) 


H. 


180 - 350 


I. 


One Year 


J. 


Two Years 


K. 


More Than Two Years 


L. 


Variable 



FIELD DEFINITION CODES fCONT. l 



C64 Descriptors 

A. Pre-Test 

B, Post-Test 

C» Systematic Program of Evaluation 
D, Criterion Referenced Tests 
E» Outline of Content Activities 

F. Teaching Suggestions 

G. Alternative Learning/Teaching Activities 

H. Suggested Resources 
L Bibliography 

J. Sample Student Materials 

Guidance Materials 
L. Inservice Training Program 
M. Stated Relationships To Other Areas 

Of The Curriculum 
N» Provisions for Individualized, Small Group, 

Large Group Instruction 
0, Out-of-School as Well as In-School 

Experiences 
P. Goals and Objectives for Learners 
R. Statement of Desired Entry Level Competencies 

of The Learner 
S. Program Organisation and Administration Guidelines 
T. Suggested Community Relations and Community 

Agency Support Programs, Which May Include 

Parent Education 
U. Student Use Materials, e.g.. Workbooks, Sample Forms, 

Tests, Etc. 
V. Other 

C65 Personnel Factors Y- 

C67 Overall Professional Judgment 

(1) Degree of Occupational Focus 

(2) Degree of Internal Consistency 

(3) Suitability to CCEM Matrix by Grade 

(4) Suitability to CCEM Matrix by Target Area 

(5) Degree of Need For Special Conditions 

(6) (Not Used) 

(7) Degree of Innovativeness, Substantive 

(8) Degree of Innovativeness, Procedural 

(9) Degree of Participant Involvement in Learning 

C69 Special Learning Conditions Y- 

Y - Staff member can use item without special training 
N • Special training required 
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V - SUMMARY OF FIELD SURVEY EFFORTS 
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SUMMARY OF FIELD CONTACTS 



AMOUNT PERCENT 



1. COMMERCIAL SUPPLIERS OF 

EDUCATIONAL MATERIALS 580 67% 



2. CHAMBER OF COMMERCE GROUPS 

OF CITIES WITH POPULATION OF 100,000 OR MORE 143 17 



3. SELECTED COMMERCIAL/INDUSTRIAL 

CORPORATIONS 130 15 



4. INDr\aDUALS AND MISCELLANEOUS 

ORGANIZATIONS _IL_ _1_ 

TOTAL 864 100% 
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CHAPTER V 



SUMMARY OF FIELD SURVEY EFFORTS 



This chapter describes the scope of survey activities and the results of the pro- 
curement efforts. 



SURVEY CONTACTS 

Extensive efforts were made to identify commercial or quasi-commercial busi- 
nesses that have career education materials available. The detailed steps and procedures 
for doing so and the system for procuring those that meet CCEM-I objectives are set forth 
in Chapter II of this report. 

As shown on the facing page, 864 contacts were made with commercial pro- 
ducers of educational materials, industrial/commercial firms. Chambers of Commerce 
and individuals who reportedly had career education materials available. Of this total, 
580 were commercial suppliers. This group was judged as the primary market target. It is 
estimated that the 580 firms represent 75 to 85 percent of the suppliers of career educa- 
tion materials and that they produce from 85 to 90 percent of the materials available. 

Of the 580 commercial suppliers contacted, 313 (54 percent) reported that 
they had materials available and were willing to cooperate as shown on the following page. 
In addition to having materials currently on the market, 128 of the 313 reportedly have 
additional materials under development. Also, 35 of the 267 suppliers that now have no 
materials available indicated that materials were being readied for the market. Hence, 
more than 340 of the 580 firms have or will soon have career education materials available 
commercially. 



MATERIALS PROCESSED 

Most of the 313 suppliers responded to the request for materials with a J^angle 
contact. However, several required five or six. In total, 1,130 contacts were made by field 
survey personnel to obtain the materials judged appropriate for assessment (see following 
pages). 

On the basis of initial field survey contacts and the suppliers' estimates of the 
number of materials that would meet CCEM-I criteria for assessment, 5,128 Product Pro- 
file Sheets were mailed. Of this number, 3,607 were returned completed. It is assumed 
that after receipt of printed materials, supphers determined that some of the materials 
they had originally identified as applicable were not. 

A review of the Product Profile Sheets resulted in the initiation of 2,766 Mat- 
erial Procurement Requests. This represented approximately 77 percent of the materials 
identified. This high percentage of applicable materials was attributable to the accuracy 
of the Product Profile screening process. 

Approximately 85 percent of the Requests for Materials resulted in their receipt, 
and all materials were assessed. Of the 2,344 units received (12,859 pieces of materials), 
Z016 were positively assessed. All positively assessed materials were indexed and abstract- 
ed for subsequent entry into the Education Research Information System (ERIC). 



SURVEY PARTICIPATION 
BY COMMERCIAL SUPPLIERS 
OF EDUCATIONAL MATERIALS 



ERIC 



NUMBE R PERCENT 
1. NO MATERIALS AVAILABLE AT THIS TIME 267 46% 



2. MATERIALS ARE AVAILABLE AND WILL 

PARTICIPATE 313 _51 



TOTAL 580 100% 



3. NO MATERIALS AVAILABLE, BUT 

MATERIALS ARE UNDER DEVELOPMENT 35 6% 



4. MATERIALS AVAILABLE, AND ADDITIONAL 

MATERIALS ARE UNDER DEVELOPMENT 128 _22. 



TOTAL 163 



SUMMARY DATA 
MATERIAL PROCUREMENT & ASSESSMENT 
PROJECT TOTALS NOVEMBER 10, 1972 



NUMBER PERCENT 



1. Supplier Contacts jj30 

2. Call Reports Inputed to EDP System 580 — 

3. R-oduct Profile Sheets Mailed to Suppliers 5128 

4. PPS^s Received 3g07 

5. Percent Received — ^0% 

6. Material Procurement Requests Initiated 2766 — 

7. Percent Initiated 77 

8. MPR's Not Initiated 841 

9. Percent Not Initiated 23 

10. Materials Received (Units) 2344 

11. As a Percent of MPR's — 85 

12. Materials Assessed (Units) 2344 — 

13. As a Percent of Materials Received — 100 

14. Materials Positively Assessed (Units)* 2016 — 

15. MPA's (4) as a Percent of MA's 

(12) (Units) _ 86 

16. Assessed Materials Exiting at X"l (Units) 200 — 

17. E(l)'s as a Percent of MA's — 9 

18. Assessed Materials Exiting at X-2 (Units) 120 128 

19. E (2) 's as a Percent of MA's — 5 

20. Index/ Abstracts Completed for PAM's 2016 — 
2L I/A's as a Percent of PAM's — 100% 



♦Includes 77 X.2J's. 
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SELECTED CHARACTERISTICS 
_ OF ASSESSED MATERIALS 



Materials applicable to specific grade levels (indicated bv suppliers) as a percent 
of the total materials procured and assessed ranged from 2 percent for kindergarten to 
nearly 14 percent for grades 12 as shown on the following page. By grade groupings 
ly 22 percent were applicable to K-6, 27 percent to grades 7 - 9, 36 percent to erad ' 
10 - 12, and 13 percent to grades 13 and 14. 



near- 

es 



ERIC 



APPLICABILITY OF 
ASSESSED MATERIAL 
BY GRADE LEVEL 



ERIC 



NUMBER OF 

GRADE MATERIALS PERCENT 



K 




258 


2.0 


1 




338 


2.6 


I* 




391 


3.0 


3 




395 


3.0 


4 




412 


3.2 


5 




437 


3.4 


6 




544 


4.3 


7 




y /o 


' 7.6 


8 




1096 


o.O 


9 




1414 


11.0 


10 




1482 


11.5 


11 




1708 


13.3 


12 




1749 


13.7 


13 




824 


6.4 


14 




836 


- 6.5 




TOTAL* 


12,859 


100% 



*Based on 2344 units of assessed materials. 



10.6 

21.5 

10.9 



27.1 



5.8 



12.9 



MEDIA CHARACTERISTICS 
OF SCREENED MATERIALS 



TYPE MA TERIAL AMOUNT P_ERCE_NT 

1. PRINTED 2270 54';^ 

2. FILM 1362 32 

3. MANIPULATIVE 110 3 

4. AUDIO-VISUAL 447 11 

TOTAL AMOUNT 4189* 100% 



*The count is greater than 2344 due to multi-media packaging of units. 
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Of the materials assessed, more than 50 percent were printed as shown on the 
facing page. Approximately one-third were film and the remainder were either audio- 
visual or manipulative. 

Suppliers were requested to indicate the selling price of materials on the com- 
pleted Product Profile Sheets. The responses are shown on the following page. The costs 
ranged from a few cents for a piece of printed material to more than $8,000 for a set of 
films. Of the 3,607 Product Profile Sheets processed, suppliers indicated that the cost of 
2,388 (66 percent of total) were available at $100 or less. Hov/ever, a relatively large 
number - - 16 percent - - did not include cost information. 



COST DISTRIBUTIONS OF 
SCREENED MATERIALS* 



COST RANGE NUMBER PERCENT 

1. UNDER $100 2388 66.2% 

2. $101 to $250 12.6 

3. $251 to $500 117 3.3 

4. $501 to $750 14 .4 

5. OVER $750 69 1.9 

6. UNKNOWN 561 15.6 

TOTAL 3607 100% 
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*Based on completed Product Profilt* Sheets. 



VI - FINDINGS AND RECOMM*^NDATIONS 
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CHAPTER VI 
FINDINGS A:^D recommendations 



The findings resulting from our engagement are presented in this chapter, fol- 
lowed by our recommendations tor CCEM actions. 



FINDINGS 

Findings resulting from the assessment of materials are presented in tabular 
form in Appendix A. Because multiple responses could be made in several judgments, 
statistical treatment is not always valid without qualification. In addition, several judg- 
ments were highly subjective. 

The Quality Control Function carefully monitored the consistency of responses 
by individual assessors and responses from different assessors on similar materials. How- 
ever, the comparative frequency, rather than number of responses, is a more reliable in- 
dicator of the applicability and quality of materials assessed against CCEM criteria. 

It is estimated that 75 to 80 percent of the suppliers of commercial career 
education materials were contacted in the Field Survey effort. A review of these contacts 
by publishing organizations and research groups indicates that these firms produce from 
85 to 90 percent of the materials available* The time and cost of obtaining the remainder 
of materials was judged excessive at this time. 

The Field Survey procedure included careful screening (through structured inter- 
view guides and a review of catalogs) of the appropriateness of materials before the deci- 
sion was made to enter them into the assessment process. Hence, 86 percent of the mat- 
erials entering the assessment process were positively assessed. Of the 14 percent of the 
materials which exited the process, 9 percent were classified as non-career education mat- 
erial or incomplete units. The remaining 5 percent exited because they did not fit the 
content-discipline, CCEM target area emphasis, CCEM consumer population or element/ 
grade level. 

Nearly 40 percent of the cooperating suppliers classified themselves as independ- 
ent commercial and an additional 20 percent as affiliated commercial organizations- Ap- 
proximately 11 percent classified their organization as a film producer or distributor. 
Of those remaining, most did not indicate type of organization. 

Only four firms declined to participate in the project. However, only 20 - 30 
percent of the cooperating companies made their material available on a complimentary 
basis. 

During the project, 33,614 pieces of material grouped in 3,607 units were 
identified. As indicated earlier, many units contained large numbers of individual pieces 
as sets or series. In addition to this relatively large amount of materials now available, 
162 of the 580 commercial suppliers (28 percent) indicated that career education mat- 
erials were under development. 

Of the materials entering the assessment process, 54 percent were printed. Ap- 
proximately 32 percent were classified as film. The remainder were manipulative and 
audio-visual. 
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The cost of materials ranged from a few cents per piece of material to SS 000 
for one set of films. However, 66 percent of the more than 3.600 units entering the a'ssess- 
ment process sell for less than $100. 

Career education ma. -rials are available commerciallv for all grade levels On the 
basis of judgments on applicability to specific grade levels, there are more than 250 
units appropriate for use in kindergartens and nearly 1.750 for grade 12. 

More than 1,000 units were determined apptopriate for each of the following 
instructional strategies: 

Discussion 1,709 

Self-instruction 1,357 

Group or panel discussions. . . 1,237 
Individual or group 

counseling i^018 

Only one unit was judged appropriate fo' -nicro-teaching and psychodrama. 

Assessors were given the opportunity to identify ea:h unit by one or more of 
Jl descriptors. A total of 3,661 responses were given for the 2,016 positively assessed 
units. The descriptor "provisions tor individualized, small group or large group instruc- 
tion was used 649 times. "Suggested community relations and community agency sup- 
port programs, which may include parent education" was determined appHcable to only 
two units. Other descriptors frequently used included: teaching suggestions - 506 suggest- 
ed resources - 454, bibliography - 374 and guidance materials - 546. 

Judgments were made regarding the degree of appropriateness {'ligh average 
k)w) of each unit to the CCEM matrix by grade level and element. Each docurnent could 
be judged applicable to a maximum of three elements and to as many grade 1, t:^.Is as ap- 
propriate. General findings are as follows: 

The frequency of applicability to grades K-14 in descending 
order were: 

- Career Awareness 

- Attitudes and Appreciations 

- Educational Awareness 

- Economic Awareness 

- Beginning Competency 

- Employability Skills 

- Decisionmaking 



Most units classified as applicable to self-awareness are for grades 
9-12. Of these, approximately 50 percent were judged average, 
25 percent high and 25 percent low 
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Educational awaren oss units are most applicable to grades 7-1 2. 
From 15 to 20 percent were judged high 

Significant numbers of career awareness u * .re available for 
all grades (at least 195). About 60 percent of tlie total jud??- 
ments indicated average applicability. However, more than 50 
percent of units for grades 4. 5 and 6 were judged low 

At least eight economic awareness units were judged appropriate 
for grades K-12. The majority of judgments of appropr:ateness 
for all grades was that materials are average. However, nearly 50 
percent of the units for grades 5 through 9 were judged to be 
low 

Only 136 decisionmakaig units were judged applicable. Most of 
these units were judged appropriate for grades 11 and 12 and 
judged as having average applicability. Of the 136, only 21 were 
determined appropriate for grades 1 through 6. However, 
approximately 50 percent of those were judged high 

Most judgments of applicability for beginning competency 
were for grades 10 • 14. More than 50 percent of the judg- 
ments were that units were of average applicability. Materials 
were judged applicable for grades 4, 5 and 6 and most of them 
were classified as having low applicability. None were assessed 
as having high applicability for grades 1 through 9 

Units determined appropriate for employability skills were 
judged rnost suitable above grade 9. Only five judgments of 
appropriateness were made for lower than grade 7 

A large number of oiaterials were judged applicable to the 
attitudes and appreciations element. Aiihough the highest 
frequency was for grades 6-12, at least 100 units were indicated 
for each grade level. However, the percent judged high is 
significantly lower than for other elements where high 
frequency was indicated. 

Judgments on the appropriateness of units to target areas by elements resulted 
in most units bemg classified as appropriate for curriculum, career information career 
preparation and guidance. Only 41 positive judgments were made for the other nine 
target areas. The element most frequently indicated was career awareness, followed by 
attitudes and appreciations. Materials determined appropriate career awareness also 
received the highest frequency of judgments of high applicability 
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These and other findings resulting from this project indicate that great effort 
has been made by commercial suppliers to provide career education materials However 
on the basis of CCEM criteria, comparatively few materials were judged highlv appropri'ite 
for a number of elements, target areas and grade levels (i.e.. self awareness materials 
below grade 9: educational awareness below grade 6; -.'areer awareness for grades 4 5 and 
6; economic awareness; decisionmaking; beginning rompetencv; employability skills- and 
attitudes and appreciations at all grade levels). This is confirmed bv judgments of over .1 
suitability to CCEM needs for materials. 74 percent of materials were rated low for 

innovativeness, substantive;" nearly 83 percent rated low for "innovativeness, procedur- 
al; and approximately 68 percent rated low for participant involvement. 

RECOMMENDATIONS 

CCEM has significant opportunities to aid users of career education materials 
in Identifying available materials and their sources. In addition, CCEM can assist com- 
mercial suppliers by maintaining an active, accurate inventory of this information This 
can be accomplished without infringing on the prerogatives of individual school officials 
or teacherr, oi the proprietary interests of commercial businesses. 

If appx-opriate, CCEM can also be of major assistance to commercial supplie-s 
of career education materials by conducting seminars to help them better understand the 
specific needs of the consumers. Many suppliers are attempting to be responsive by mere- 
ly changing titles and/.jr mserting chapters in traditional materials. Most of these materials 
only minimally meet CCEM-I objectives. In part, this has been done because of the time 
requir-d to develor. new materials. However, several suppliers are unclear about the 
definition, theory and desired nature, scope and use of career education materials. 

Specific recommendations are as follows: 

1. Place the resukii the project(s) in the ERIC system. 

2. Establish and make available necessary resources for continuing the pro- 
cess of identifying, procuring, assessing, abstracting and indexing career education 
materials. 

3. Ficre high emphasis on screening materials through careful review of 
catalogs to determin- their probable appropriateness before issuing Material Procurement 
Requests. This will substantially reduce the resource and time requirements associated 
with each positively assessed unit. 

4. Adopt and implement systems used in this project and make modifications 
and refinements deemed necessary to keep them efficient and operable at the least re- 
quired cost. 

5. Maintain and expand relationships with supplier as appropriate. This re- 
quires prompt and responsive handling. 

A ! u u^' °° change the CCEM matrixes or definitions for at least one year 
Although refinements might improve the structure, they can be confusing to both con- 
sumers and suppliers. 



7. Distribute detailed * -ndings of this projt-ci id LK.a s as soon a^ {)(:svih!t'. 

8. Befjin a formal plan of action to convey CCMM's conc ept (^f can-t'i- educa- 
tion and provide specific direction in terms of required types of commerciallv produn^d 
materials. 

In summary. CCEM - - in conjunction with other education^d agencies, ins* it 
tions, organizations and commercial suppliers - - has a i:nique opportunity tc^ assume 
leadership in improving the nature, scope and use of educational materials. Much of its 
success ir doing so will depend on takinj? immediate advantage of the comprehensive data 
assembled in this project and its earlier companion study of the availability of non-com> 
mercial material. 



